Supervised classification in high-dimensional space: geometrical, statistical, and asymptotical properties of multivariate data

As the number of spectral bands of high spectral resolution data increases, the capability to detect more detailed classes should also increase, and the classification accuracy should increase as well. Often the number of labeled samples used for supervised classification techniques is limited, thus limiting the precision with which class characteristics can be estimated. As the number of spectral bands becomes large, the limitation on performance imposed by the limited number of training samples can become severe. A number of techniques for case-specific feature extraction have been developed to reduce dimensionality without loss of class separability. Most of these techniques require the estimation of statistics at full dimensionality in order to extract relevant features for classification. If the number of training samples is not adequately large, the estimation of parameters in high dimensional data will not be accurate enough. As a result, the estimated features may not be as effective as they could be. This suggests the need for reducing the dimensionality via a preprocessing method that takes into consideration high dimensional feature space properties. Such reduction should enable the estimation of feature extraction parameters to be more accurate. Using a technique referred to as Projection Pursuit, such an algorithm has been developed. This technique is able to bypass many of the problems of the limitation of small numbers of training samples by making the computations in a lower dimensional space, and optimizing a function called the projection index. A current limitation on this method is that as the number of dimensions increases, it is highly probable to find a local maximum 1 Work reported herein was funded in part by NASA Grant NAGW-3924. 2 Corresponding author. Projection Pursuit in Hyperspectral Data Analysis Jimenez & Landgrebe 2 Printed November 23, 1999 of the projection index that does not enable one to fully exploit hyperspectral data capabilities. A method to estimate an initial value that can lead to a maximum that increases significantly the classification accuracy will be presented. This method leads also to a high dimensional version of a feature selection algorithm, which requires significantly less computation than the normal procedure.